Extremal Optimization at the Phase Transition of the 3-Coloring Problem 
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We investigate tlie pliase transition of tiie 3-coloring problem on random graphs, using tlie ex- 
tremal optimization lieuristic. 3-coloring is among the hardest combinatorial optimization problems 
and is closely related to a 3-state anti-ferromagnetic Potts model. Like many other such optimiza- 
tion problems, it has been shown to exhibit a phase transition in its ground state behavior under 
variation of a system parameter: the graph's mean vertex degree. This phase transition is often 
associated with the instances of highest complexity. We use extremal optimization to measure the 
ground state cost and the "backbone", an order parameter related to ground state overlap, aver- 
aged over a large number of instances near the transition for random graphs of size n up to 512. 
For graphs up to this size, benchmarks show that extremal optimization reaches ground states and 
explores a sufficient number of them to give the correct backbone value after about 0(n'^'^) update 
steps. Finite size scaling gives a critical mean degree value Oc = 4.703(28). Furthermore, the explo- 
ration of the degenerate ground states indicates that the backbone order parameter, measuring the 
constrainedness of the problem, exhibits a first-order phase transition. 

PACS numbers: 02.60.Pn, 05.10.-a 75.10.Nr, 
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I. INTRODUCTION 



The most chahenging instances of computational prob- 
lems are often found near a critical threshold in the prob- 
lem's parameter space where certain characteristics 
of the problem change dramatically. One such problem, 
already discussed in Ref. 0, S S Si is the 3-coloring 
problem. Consider a random graph 6] having n vertices 
and m edges placed randomly among all possible pairs 
of vertices. The number of edges emanating from each 
vertex is then Poisson-distributed around a mean degree 
a = 2m/ n. To 3-color the graph, we need to assign one of 
three colors to each vertex so as to minimize the number 
of "monochromatic" edges, i.e., those connecting vertices 
of the same color. In particular, we may want to decide 
whether it is possible to make an assignment without any 
monochromatic edges using, for instance, a backtracking 
assignment procedure. Typically, if the mean degree a 
is low (for example, when each vertex most likely has 
fewer than 3 neighbors) , one quickly finds a perfect color- 
ing. If the mean degree is high, one soon determines that 
monochromatic edges are unavoidable after fixing just a 
small number of vertices. At an intermediate mean de- 
gree value, however, some graphs are perfectly colorable 
while others are not. In that case, for each instance one 
must inspect many almost-complete colorings, most of 
which fail only when trying to assign a few final vertices, 
before colorability can be decided^, Q ■ For increasing 
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n, the regime of mean degree values a for which the de- 
cision problem is hard becomes narrowly focused, while 
the computational complexity of the backtracking algo- 
rithm within this regime grows faster than any power of 
n, signs of the impending singularity associated with a 
phase transition. 

Such findings have spawned considerable interest 
among computer scientists and statistical physicists alike. 
On one hand, there appear to be close links to the prop- 
erties of spin glass systems Q . Using replica symmetry 
breaking, it was recently argued 0,0 that 3-coloring un- 
dergoes a colorability transition at Ofcrit = 4.69, heralded 
by the spontaneous emergence at a = 3.35 of a sizable 
3-core [lOj that becomes over-constrained at the transi- 
tion. This analysis shows, furthermore, that the hardest 
instances to decide are located between a clustering tran- 
sition at a ~ 4.42 and acrit- On the other hand, attempts 
have been made to relate the nature of the phase transi- 
tion to complexity classifications developed by computer 
scientists for combinatorial problems llj: it has been 
suggested that NP-complete problems, which are hard 
to solve, display a first-order phase transition while eas- 
ier problems lead to a second-order transition [l^ . Such 
a relation, while intriguing and suggestive, is bound to 
be questionable in light of the fact that these phase tran- 
sitions are based on a notion of average-case complexity 
(for instance, 3-coloring averaged over the ensemble of 
random graphs), distinct from the notion of worst-case 
complexity used by computer scientists to define NP- 
completeness and other such complexity theoretic 
categories. In fact, it currently appears that a first-order 
transition is merely an indicator for the complexity of cer- 
tain types of algorithms: local searches. A model prob- 
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lem with a discontinuous transition, i^-XORSAT p^lTsl l. 
can be solved by a fast global algorithm yet it is ex- 
tremely hard for local search or backtracking assignment 
algorithms. 

In this paper we consider the 3-coloring problem men- 
tioned above. The problem is among the hardest com- 
binatorial optimization problems, making it difficult to 
study the asymptotic properties of its phase transition 
with exact methods. It is also of considerable interest 
in its own right as a model for many practical optimiza- 
tion problems 11], and it is of some physical relevance 
due to its close relation to Potts anti-ferromagnets [T^ . 
Some aspects of the 3-coloring phase transition have pre- 
viously been explored [^^|5|. In particular, Culberson 
and Gent |5| have studied the phase transition for random 
graphs with exact methods by "growing" random graphs 
of size n < 225, sequentially adding random edges to 
an existing graph to increase a, and checking along the 
way whether the graph is still colorable. Once a graph 
becomes uncolorable, it is discarded from the list of grow- 
ing graphs, so the set of graphs becomes increasingly less 
representative of the ensemble when passing through the 
transition. In the process, these authors have evaluated 
the constrainedness of the variables in the graph, study- 
ing in detail many aspects of the approach to the tran- 
sition. However, the main quantity they measure, called 
the "spine" ^3 ; is in general an upper bound on the or- 
der parameter we consider here, since contributions from 
uncolorable graphs are neglected. 

We investigate the properties near the phase transition 
by applying an o ptim ization heuristic called extremal op- 
timization (EO) p','!?!. EO was recently introduced as 
a general-purpose optimization method based on the dy- 
namics of driven, dissipative systems Our study 
shows that EO is capable of determining many ground- 
state properties efficiently, even at the phase transition 
[2l[ . EO performs a local neighborhood search that does 
not get stuck in local minima but proceeds to explore 
near-optimal configurations broadly. Hence, it is partic- 
ularly well suited to measure global properties of the con- 
figuration space. Here, we use it to estimate the "back- 
bone" , an overlap property between the highly degener- 
ate ground state configurations that provides a more con- 
venient order parameter than measuring mutual overlaps 
of all ground states While EO is not exact, bench- 
mark comparisons with exactly-solved, large instances 
justify our confidence in its results. Our biggest errors 
originate from the lack of statistics at large n. 

Our results indicate that the transition in the backbone 
size is of first order, though with only a small disconti- 
nuity. In fact, the discontinuity does not arise uniformly 
for all graphs in the ensemble, but is due to a fraction of 
instances that have a strong backbone of a characteristic 
size while the rest have hardly any backbone at all. 

Using the procedure of Ref. to control the qual- 
ity of finite size scaling for the ground state cost func- 
tion, we estimate the location of the transition as Ofcrit ~ 
4.703(28), where the numbers in parentheses denote the 



statistical error bar in the final digits. This is consis- 
tent with the presumably correct value of a = 4.69 
given b^ replica symmetry breaking methods (see also 
Refs. 3, ^ for earlier estimates). We measure the size 
of the scaling window as n"^/"^ with v = 1.43(6), close 
to the value of 1.5 estimated for 3-SAT |2^, although it 
may be that trivial n~^/^-fiuctuations from the variables 
not belonging to the 3-core |J0] dominate at much larger 
n than considered here [2^ . 

In the following section, we introduce the problem of 
3-coloring in more detail and discuss the relevant observ- 
ables we measure in order to analyze the phase transi- 
tion. In Sec, mil we discuss our EO implementation and 
its properties. In Sec. II VI we present the results of our 
measurements, and we conclude with Sec. 



II. 3-COLORING OF RANDOM GRAPHS 

A random graph 6] is constructed from a set of n ver- 
tices by assigning an edge to m = an/2 of the (2) pairs 
of vertices with equal probability, so that a is the average 
vertex degree. Here, we will only consider the regime of 
"sparse" random graphs where m = 0(n) and a = 0(1). 
The goal of graph coloring is to label each vertex with a 
different color so as to avoid monochromatic edges. 

Three different versions of the coloring problem are of 
interest. First, there is the classic problem of determin- 
ing the "chromatic number" for a given graph, i.e., the 
minimum number of colors needed to color the graph 
while avoiding monochromatic edges. It is very diffi- 
cult to devise a heuristic for this problem In the 
other two versions, we are given a fixed number K of 
colors to select from. The decision problem, K-COL, 
addresses the question of whether a given graph is col- 
orable or not. Finally, the optimization problem, MAX- 
K-COL, tries to minimize the number of monochromatic 
edges (or equivalently, maximize the number of non- 
monochromatic edges, hence its name). Clearly, if we 
define the number of monochromatic edges as the "cost" 
or "energy" of a color assignment, determining whether 
the minimal cost is zero or non-zero corresponds to solv- 
ing the decision problem K-COL, so finding the actual 
cost of the ground state is always at least as hard. Much 
of the discussion regarding the complexity near phase 
transitions in the computer science literature is focused 
on the decision problem 0, . From a physics perspec- 
tive, it seems more intuitive to examine the behavior of 
the ground states as one passes the transition. Accord- 
ingly, we will focus on the MAX-iC-COL problem in this 
paper. 

All these versions of coloring are NP-hard and 
thus computationally hard in the worst case. To de- 
termine exact answers would almost certainly require a 
computational time growing faster than any power of n. 
Thus, extracting results about asymptotic properties of 
the problems is a daunting task, calling for the use of 
accurate heuristic methods, as discussed in the following 



3 



section. 

The control parameter describing our ensemble of in- 
stances is the average vertex degree a of the random 
graphs. Constructing an appropriate order parameter 
to classify the transition is less obvious. The analogy to 
spin-glass theory suggests the following reasoning. 

In a homogeneous medium possessing a single pure equi- 
librium state, the magnetization provides the conjugate 
field to analyze the ferromagnetic transition. For our 
3-coloring problem, the disorder induced by the random 
graphs leads to a decomposition into many coexisting but 
unrelated pure states with a distribution of magnetiza- 
tions. Since the colors correspond to the spin orientations 
in the related Potts model, we need in principle to de- 
termine, for each graph, the overlap between all pairs of 
ground state colorings. Finally, this distribution has to 
be averaged over the ensemble. To simplify the task, one 
can instead extract directly the "backbone" , which is the 
set of variables that take on the same state in all possi- 
ble ground state colorings of a given instance • But 
even determining the backbone is a formidable undertak- 
ing: it requires not only finding a lowest cost coloring but 
sampling a substantial number of those colorings for each 
graph, since the ground state entropy is extensive. 

Another level of difficulty arises due to the invariance 
of the ground states under a global color permutation. 
Thus, in the set of all ground states, each vertex can take 
on any color and the backbone as defined above is empty. 
To avoid this triviality, one may redefine the backbone in 
the following way. Instead of considering individual ver- 
tices, consider all pairs of vertices that are not connected 
by an edge Define the pair to be part of the frozen 
backbone if its vertices are of like color (monochromatic) 
in all ground state colorings, so that the presence of an 
edge there would necessarily incur a cost. Define the pair 
to be part of the free backbone if its vertices are of unlike 
color (non-monochromatic) in all ground state colorings, 
so that the presence of an edge there would not incur any 
cost. Since the fraction of pairs that belong to the frozen 
backbone measures the constrainedness of an instance, it 
is the relevant order parameter. We have also sampled 
the free backbone. As shown in Sec. IIVI both seem to 
exhibit a first-order transition, though the jump for the 
frozen backbone is small. 

By definition, the location of the transition is deter- 
mined through a (second-order) singularity in the cost 
function C: the cost is asymptotically vanishing below 
the transition, it is continuous at the transition, and 
above it is always non-zero. We have therefore measured 
the ground state cost, averaged over many instances, for 
a range of mean degree values a and sizes n. 



III. EXTREMAL OPTIMIZATION 

To investigate the phase transition in 3-COL, we em- 
ploy the extremal optimization heuristic (EO) [ig- The 
use of a heuristic method, while only approximate, allows 



us to measure observables for much larger system sizes n 
and with better statistics then would be accessible with 
exact methods. We will argue below that we can obtain 
optimal results with sufficient probability that even sys- 
tematic errors in the exploration of ground states will be 
small compared to the statistical sampling error. 

Our EO implementation is as follows. Assume we 
are given a graph with a (however imperfect) initial as- 
signment of colors to its vertices. Each vertex i has ai 
edges to neighboring vertices, of which < gi < ai are 
"good" edges, i.e., to neighbors of a different color (not 
monochromatic). We define for each vertex a "fitness" 

A. = ^e[0,i], (1) 

Oil 

and determine a permutation 11 (not necessarily unique) 
over the vertices such that 

An(i) < An(2) < • ■ • < An(„)- (2) 

At each update step, EO draws a number k from a dis- 
tribution 

P{k) - fc-^ {l<k< n) (3) 

with a bias toward small numbers. A vertex i is selected 
from the ordered list in Eq. (|2Jl according to its "rank" fc, 
i.e., i = Il{k). Vertex i is updated unconditionally, i.e., it 
always receives a new color, selected at random from one 
of the other colors. As a consequence, vertex i and all 
its neighbors change their fitnesses A and a new ranking 
n will have to be established. Then, the update process 
starts over with selecting a new rank fc, and so on until 
some termination condition is reached. Along the way, 
EO keeps track of the configuration with the best coloring 
it has visited so far, meaning the one that minimizes the 
number of monochromatic edges, C = '^^iai — gi)/^. 

Previous studies have found that EO obtains near- 
optimal solutions for a variety of hard optimization prob- 
lems for a carefully selected value of r [23, 113 ■ 
For 3-COL, initial trials have determined that best re- 
sults are obtained for the system sizes n = 32, 64, . . . , 512 
at a (fixed) value of r « 2.2. This rather large value of 
T '28] helps explore many low-cost configurations effi- 
ciently; if we merely wanted to determine low-cost solu- 
tions, larger values of n could have been reached more 
efficiently at a smaller value of r. 

It should be noted that our definition of fitness does 
not follow the generic choice Ai = gi/2 that would give a 
total configuration cost of C = const. — A.^. While this 
formulation sounds appealing, and does produce results 
of the same quality, our choice above produces those same 
results somewhat faster; there appears to be some advan- 
tage to treating all vertices, whose individual degrees ai 
are Poisson-distributed around the mean a, on an equal 
footing. Furthermore, our implementation limits itself to 
partially sorting the fitnesses on a balanced heap [Tsl l. 
rather than ranking them perfectly as in Eq. ^ . In this 
way, the computational cost is reduced by a factor of n 
while performance is only minimally affected [lSj |. 
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A. Measuring the backbone 

The backbone, described in Sec.^ is a collective prop- 
erty of degenerate ground states for a given graph. Thus, 
in this study we are interested in determining not only 
the cost C of the ground state, but also a good samphng 
of all possible ground state configurations. Local search 
with EO is ideally suited to probe for properties that are 
broadly distributed over the configuration space, since 
for small enough t it does not get trapped in restricted 
regions. Even after EO has found a locally minimal cost 
configuration, it proceeds to explore the configuration 
space widely to find new configurations of the same or 
lower cost, as long as the process is run. 

Against these advantages, one must recognize that EO 
is merely a heuristic approximation to a problem of ex- 
ponential complexity. Thus, to safeguard the accuracy 
of our measurements, we devised the following adaptive 
procedure. For each graph, starting from random initial 
colorings, EO was run for update steps, using a min- 
imum of 5 different restarts. For the lowest cost seen so 
far, EO keeps a buffer of up to n/4 most recently visited 
configurations with that cost. If it finds another configu- 
ration with the same cost, it quickly determines whether 
it is already in the buffer. If not, EO adds it on top of 
the buffer (possibly "forgetting" an older configuration 
at the bottom of the buffer). Thus, EO does not keep 
a memory of all minimal cost configurations seen so far, 
which for ground states can have degeneracies of > 10^ 
even at n = 64. Instead of enumerating all ground states 
exhaustively, we proceed as follows. When EO finds a 
new, lowest cost configuration, it assumes initially that 
all pairs of equally colored vertices are part of the frozen 
backbone and all other pairs are part of the free back- 
bone. If another configuration of the same cost is found 
and it is not already in the buffer, EO checks all of the 
pairs in it. If a pair has always been frozen (free) be- 
fore and is so now, it remains part of the frozen (free) 
backbone. If a pair was always frozen (free) before and 
it is free (frozen) in this configuration, it is eliminated 
from both backbones. If a pair has already been elim- 
inated previously, no action is taken. In this way, cer- 
tain ground-state configurations may be missed or tested 
many times over, without affecting the backbones signif- 
icantly. 

Eventually, even if new and unrecognized configura- 
tions of the lowest cost are found, no further changes 
to either backbone are likely to occur. This fact moti- 
vates our adaptive stopping criterion for EO. Assume the 
current backbone was last modified in the rpth restart. 
Then, for this graph EO restarts for a total of at least 
r = tq + •max{rQ, 5} times, terminating only when there 
has been no updates to the backbone over the previous 
max{rQ, 5} restarts. Of course, every time a new, lower- 
cost configuration is found, the buffer and backbone ar- 
rays are reset. Ultimately, this procedure leads to adap- 
tive runtimes that depend on the peculiarities of each 
graph. The idea is that if the lowest cost state is found 



in the first start and the backbone does not change over 
5 more restarts, one assumes that no further changes to 
it will ever be found by EO. However, if EO keeps updat- 
ing the backbone through, say, the 20th restart, one had 
better continue for 20 more restarts to be confident of 
convergence. The typical number of restarts was about 
10, while for a few larger graphs, more than 50 restarts 
were required. 

A majority of our computational time is spent merely 
confirming that the backbones have converged, since dur- 
ing the final TOaa;{ro,5} restarts nothing new is found. 
Nevertheless, EO still saves vast amounts of computer 
time and memory in comparison with exact enumera- 
tion techniques. The trade-off lies in the risk of missing 
some lowest cost configurations, as well as in the risk of 
never finding the true ground state to begin with. To get 
an estimate of the systematic error resulting from these 
uncontrollable risks, we have benchmarked our EO im- 
plementation against a number of different exact results. 

First, we used a set of 700 explicitly 3-colorable graphs 
over 7 different sizes, n = 75, 100, . . . , 225 (100 graphs 
per value of n) at a = 4.7, kindly provided by J. Cul- 
berson, for which exact spine values [l7j were found as 
described in Ref. 5]. For colorable graphs such as these, 
the spine is identical to the backbone. Our EO imple- 
mentation correctly determined the 3-colorability of all 
but one graph, and reproduced nearly all backbones ex- 
actly, regardless of size n. For all graph sizes, EO failed 
to locate colorable configurations on at most 5 graphs out 
of 100, and in those cases overestimated either backbone 
fraction by less than 4%. Only at n = 225 did EO miss 
the colorability of a single graph to find C = 1 instead, 
thereby underestimating both backbones. 

In a different benchmark, containing colorable as well 
as uncolorable graphs, we generated 440 random graphs 
over 4 different sizes, n = 32,64, 128,256 and 11 differ- 
ent mean degree values a — 4.0, 4.1, . . . , 5.0 (10 graphs 
per value of n and a). We found the exact minimum 
cost and exact fraction of pairs belonging to the back- 
bone for these graphs, by removing edges until an ex- 
act branch-and-bound code due to M.A. Trick 3Q] deter- 
mined 3-colorability. For example, finding that a graph 
had a ground state cost of C = 2 involved considering 
all possible 2-edge removals until a remainder graph was 
found to be 3-colorable. We then added edges to ver- 
tex pairs in this remainder graph, checking whether the 
graph stayed colorable: if so, that pair was eliminated 
from the frozen backbone. Likewise, we merged vertex 
pairs, checking whether the graph stayed colorable: if so, 
that pair was eliminated from the free backbone. Finally, 
we would repeat the procedure on all colorable 2-edge- 
removcd remainder graphs, potentially eliminating pairs 
from the frozen and free backbones each time. Compar- 
ing with the exact testbed arising from this procedure 
shows that, for all graphs, EO found the correct ground 
state cost. Moreover, EO overestimated the frozen back- 
bone fraction on only 4 graphs out of 440 (2 at n = 128 
and 2 dX n — 256, in both cases at a = 4.6), and by 
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at most 0.004. This leads to a predicted systematic er- 
ror that is at least an order of magnitude smaller than 
the statistical error bars in the results we present later. 
EO's free backbone results were not of as good quality, 
overestimating the backbone fraction on 36 graphs out of 
440, by an average of 0.003 though in one case {n — 256, 
a = 4.6) by as much as 0.027. The resulting systematic 
error, however, is still small compared to the statistical 
error bars in our main results. 

It is also instructive to study the running times for 
EO, how they scale with increasing graph size, and how 
they compare with the exact algorithm we have used for 
benchmarking. In our EO implementation, we have mea- 
sured the average number of update steps it took (1) to 
find the ground state cost for the first time, (iGs)j and 
(2) to sample the backbone completely, (^bb). (Note that 
tBB, corresponding to tq in Sec. lIII Al is always less than 
half the total time spent to satisfy the stopping criterion 
for an EO run described above.) Both ^qs smd Ibb can 
fluctuate widely for graphs of a given n and a, especially 
when C > 0. However, since our numerical experiments 
involve a large number of graphs, the average times (tcs) 
and (tBB) are reasonably stable. Furthermore, (tcs) and 
(^bb) show only a weak dependence on a, varying by no 
more than a factor of 2: (tcs) increases slowly for increas- 
ing a, and (tBB) has a soft peak at ac fsi'l. Thus, for each 
n we average these times over a as well, leading only to a 
slight increase in the error bars (on a logarithmic scale). 
We have plotted the average quantities in Fig. ^ on a 
log-log scale as a function of n. It suggests that the time 
EO takes to find ground states increases exponentially 
but very weakly so. Once they are found, EO manages 
to sample a sufficient number of them in about 0{n^'^) 
updates to measure the backbone accurately. 

By contrast, one cannot easily quantify the scaling 
behavior of running times for the exact branch-and- 
bound benchmarking method. As ground state cost in- 
creases, the complexity of the method quickly becomes 
overwhelming, and rules out using it to measure aver- 
age quantities with any statistical significance. Clearly, 
branch-and-bound itself has exponential complexity for 
determining colorability. For the sizes studied here, how- 
ever, the exponential growth in n appears in fact sub- 
dominant to the 0{n^'^^) complexity of evaluating the 
backbone for a graph with non-zero ground state costs. 
When C = 1 or 2, the combinatorial effort is manageable, 
but a.t n — 256, graphs just at the transition (a = 4.70) 
reach C > 3 and the algorithm takes weeks to test all 
remainder graphs. From this comparison, one can appre- 
ciate EO's speed in estimating the backbone fractions, 
however approximate! 



IV. NUMERICAL RESULTS 

With the EO implementation as described above, we 
have sampled ground state approximations for a large 
number of graphs at each size n. In particular we have 
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FIG. 1: Log-log plot of the average time (tcs) to reach the 
lowest cost state found (circles) and {Ibb) to sample the back- 
bone (squares), in units of EO-update steps, as a function of 
system size n. The dashed line, O.OOSn"^'^ exp(0.004n) gives a 
reasonable fit to (tcs) (after having assumed the n^'^ power 
law for the fit), and the dash-dotted line, 0.3n^'^, is a fit 
obtained for (tBB). (Taking these crude fits at face value, 
merely reaching the first good ground state approximations 
would begin to dominate the runtime at about n ~ 10'^). 



considered, over a range of a, 100000 random graphs of 
size n = 32, 10 000 of size n = 64, 4 000 of size n = 128, 
and 1 000 of size n = 256. By averaging over the lowest 
energies found for these graphs, we obtain an approxima- 
tion for average ground state costs (C) as a function of a 
and n, as shown in Fig. |21 We have also sampled 160 in- 
stances of size n = 512, which provided enough statistics 
for the backbone though not for the ground state costs. 
With a finite size scaling ansatz 



(C) ~ n^f (a - Qfcrit) n 



(4) 



systematically applied "2^, it is possible to extract pre- 
cise estimates for the location of the transition ac and 
the scaling window exponent v. In the scaling regime, 
one might assume that the cost for the fixed argument 
of the scaling function is independent of the size, i.e., 
(5 = 0, indicated by the fact that for all values of n the 
cost functions cross in virtually the same point. Hence, in 
results we have previous reported 19] , we obtained what 
appeared to be the best data collapse by fixing 5 = 
and choosing acrit = 4.72(1) and v = 1.53(5), with the 
error bars in parentheses being estimates based on our 
own assessment of the data collapse. But a more careful 
automated fit to our data, provided to us by S.M. Bhat- 
tacharjee, gives 5 ~ —0.001(3), acrit = 4.703(28), and 
V = 1.43(6) with a tolerance level of 77 = 1% (see 
Ref. [12). While these fits are consistent with our previ- 
ous results, they are also consistent with and much closer 
to the presumably exact result of acrit = 4.69 ... 01 and 
the error estimates are considerably more trustworthy. 

The scaling window is determined by two competing 
contributions: for the intermediate values of n accessi- 
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FIG. 2: Plot of the average cost as a function of the vertex 
degree a. After correct finite size scaling, the data collapses 
onto a single scaling function, as shown in the insert. The fit 
gives acrit ~ 4.70, marked by a vertical line. 



ble in this study it is dominated by nontrivial contribu- 
tions arising from the correlations amongst the variables, 
which yields v w 1.43(6), similar to satisfiability prob- 
lems 23j. However, for sufficiently large n, Wilson 
has shown that i' > 2, due to intrinsic features of the 
ensemble of random graphs. The argument may be para- 
phrased as follows. Since a = 0{1) and vertex degrees 
are Poisson-distributed with mean a, a finite fraction of 
vertices in a random graph have degrees 0, 1, or 2 (those 
not belonging to the 3-core ) and thus cannot possibly 
cause monochromatic edges. But this finite fraction itself 
undergoes (normal) ~ fluctuations, and these fluc- 

tuations limit the narrowing of the cost function's scaling 
window at large n. Such variables make up about 15% 
of the total near acrit j so we estimate the crossover to 
occure at n^^^'^ ^ 0.15n^^/^ or n « 10^, assuming all 
other constants to be unity. 

Our next main result is the estimate of the backbone 
near the phase transition, as described in Sec. IIII Al Wc 
have sampled the frozen and the free backbones 5] sep- 
arately. Our results show the fraction of vertex pairs in 
each backbone, and are plotted in Fig. |21 For the free 
backbone, consistent with our definition, we do not in- 
clude any pairs that are already connected by an edge. 
Although they make up only 0{l/n) of the pairs, the in- 
clusion of these would cause a significant finite size effect 
when the backbone is small, and only by omitting them 
does the free backbone vanish for a < ctcrit- In princi- 
ple, according to our definition one should also be sure 
to exempt from the frozen backbone any pairs that are 
connected by a monochromatic edge in all ground state 
configurations, but at acrit their impact on the backbone 
is only 0(l/n^). 

As Fig. shows, both backbones appear to evolve to- 
ward a discontinuity for increasing n. The backbone frac- 
tion comes increasingly close to vanishing below acrit , fol- 
lowed by an increasingly steep jump and then a plateau 




4.2 4.4 4.6 4.8 5.0 

Vertex Degree a 

FIG. 3: Plot of the frozen (top) and the free (bottom) back- 
bone fraction as a function of the vertex degree a. The critical 
point Qfcrit ~ 4.70 is indicated by a vertical line. 



that, to within statistical noise, appears stable at large n. 
The height of the plateau at acrit suggests that on aver- 
age about 6% of all pairs are frozen and close to 20% are 
free, with both values rising further for increasing degree. 
The "jump" in the frozen backbone is somewhat smaller 
than that in the free backbone, adding a higher degree 
of uncertainty to that interpretation, although still well 
justified within the error bars. Indeed, given the consid- 
erable ground state degeneracies, it would be surprising 
if the frozen backbone were large. 

A more detailed look at the data (Fig. ^ suggests that 
the distribution of frozen backbone fractions for individ- 
ual instances is bimodal at the transition, i.e., about half 
of the graphs have a backbone well over 10% while the 
other half have no backbone at all, leading to the average 
of 6% mentioned above. Furthermore, there appears to 
be some interesting structure in the backbone disconti- 
nuity, which may be significant beyond the noise. Note 
in Fig.Othat for larger n, the increase of the frozen back- 
bone stalls or even reverses right after the jump before 
rising further. This property coincides with the emer- 
gence of non-zero costs in the ground state colorings (see 
Fig.[2Jl. The sudden appearance of monochromatic edges 
seems initially to reduce the frozen backbone fraction: 
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FIG. 4: Plot of the typical frozen (top) and free (bottom) 
backbone probability, obtained here from the n = 128 graphs. 
For mean degree values a < Uait, graphs are almost certain 
to have a vanishing backbone, both frozen and free. Above 
Cfcrit a majority of graphs still do not exhibit any backbone, 
but a finite fraction of graphs display a sizable backbone frac- 
tion, clustered at a characteristic size. The average backbones 
plotted in Figs.|3|represent the average of these apparently bi- 
modal distributions. This qualitative picture appears to hold 
for increasing n, although n > 128 data are somewhat noisy. 

typically there are numerous ways of placing those few 
edges, often affecting the most constrained variables pairs 
and eliminating them from the frozen backbone. Similar 
observations have been made by Culberson Accord- 
ing to this argument, only the frozen backbone should 
exhibit such a stall (or dip). Indeed, Fig. |31 indeed shows 
a less hindered increase in the free backbone, though the 
difference there may be purely due to statistical noise. 

V. CONCLUSIONS 

We have considered the phase transition of the MAX- 
3-COL problem for a large number of instances of random 



graphs, of sizes up to n = 512 and over a range of mean 
degree values a near the critical threshold. For each in- 
stance, we have determined the fraction of vertex pairs 
in the frozen and free backbones, using an optimization 
heuristic called extremal optimization (EO) 18J. Based 
on previous studies , EO is expected to yield an excel- 
lent approximation for the cost and the backbone. Com- 
parisons with a testbed of exactly-solved instances sug- 
gest that EO's systematic error is negligible compared to 
the statistical error. 

Using a systematic procedure for optimizing the data 
collapse in finite size scaling p3 |. we have argued that 
the transition occurs at a yit = 4.703(28), consistent 
with earlier results H, ^ H, Ell as well as with a recent 
replica symmetry breaking calculation yielding 4.69 0. 
We have also studied both free and frozen backbone frac- 
tions around the critical region. A simple argument [l^ 
demonstrates that below the critical point the backbone 
fraction always vanishes for large n. At and above the 
critical point, neither backbone appears to vanish, sug- 
gesting a first-order phase transition. This is in close 
resemblance to if-SAT for K = 3 indeed, both are 
computationally hard at the threshold. 

Even though the backbone is defined in terms of 
minimum-cost solutions, its behavior appears to corre- 
late more closely with the complexity of finding a zero- 
cost solution (solving the associated decision problem) at 
the threshold. One possible explanation is that instances 
there have low cost, so finding the minimal cost is only 
polynomially more difficult than determining whether a 
zero-cost solution exists. Interestingly, our 3-coloring 
backbone results mirror those found for the spine , an 
upper bound on the backbone that is defined purely with 
respect to zero-cost graphs. The authors of that study 
speculate that at the threshold, although the spine is 
discontinuous, the backbone itself might be continuous. 
Our results contradict this speculation, instead provid- 
ing support for a relation — albeit restricted — between 
backbone behavior and average-case complexity. 
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